Analysis of Multiple Sign Language Recognition Using Leap Motion Sensor

Sign acquisition was mainly done using camera or sensor. Due to the invention of some advance devices like Leap Motion Sensor and Kinect the researchers have experienced new horizon for making Sign Language Recognition system more accurate. In this paper, an analysis of different Neural Networks for three sign languages is presented. Many experiments are performed for measuring the performance of NN. Sign Language recognition system is developed for three sign languages namely ASL, CSL and ISL using Leap Motion Sensor. Leap Motion sensor overcomes the major issues in real time environment like background, lightening condition, and occlusion. The leap motion sensor captures the hand gesture and gives finger position in 3D format. The positional information of five finger tips along with center of palm for both the hand is used to recognize sign posture. Signs are performed using one hand mainly and some signs in ISL are performed using both the hands. While experimentation it is observed that by keeping Leap Motion sensor little inclined, the depth information was more accurate and sign was properly visible in skeleton form. So 10 degree inclination is fixed up for sensor. So that depth information is properly extracted. The focus was mainly on Finger spell recognition so dynamic signs are not considered. 32 signs of ASL, 34 signs of CSL and 33 signs of ISL are recognized. Database is created using number of users belongs to different age, sex and region. Different Neural Network classifiers like MLP, GFF and SVM are trained and tested. For ASL recognition maximum classification accuracy as 90% is obtained on CV dataset using MLP NN. For CSL recognition it was 93.11% on CV dataset using SVM NN. In ISL recognition, maximum classification accuracy of 96.36% is obtained on CV dataset using GFF NN. Although Leap Motion sensor tracks both the hand accurately it can’t track non manual signs which involve other body parts and facial expressions. Keywords— American Sign Language (ASL), Indian Sign Language (ISL), Chinese Sign Language (CSL).